Lecture notes on Stopping Times

Markov chains are awesome and can solve some important computational problems. A classic example is computing the volume of a convex body, where Markov chains and random sampling provide the only known polynomial time algorithm. Arguably the most important question about a Markov chain is how long to run it until it converges to its stationary distribution. Generally, a chain is run for some fixed number of steps, until the current point is provably within some threshold distance from the target distribution. However, this ignores the information we gain as the chain takes its steps, which could potentially be beneficial for determining convergence. For instance, we might converge much faster than the mixing time suggests, or perhaps the mixing time is unknown, in which case we have no a priori way to know how long to run the Markov chain. So instead, perhaps we observe some quantities of the walk, and then announce convergence once some condition is met. Such algorithms which, at a high level, use the walk to determine convergence are referred to as stopping rules. Also note that we can use stopping rules to create distributions other than the stationary distribution of the Markov chain. A trivial example of such a stopping rule is to stop the walk when we reach a current state x. We will see other more interesting stopping rules that can generate complex distributions over the state space.